The Optimal Allocation of Behavior Utility and Adaptation to Constraint

The last chapter looked at feeding as a typical motivational system and examined the effects of variables such as palatability, diet dilution, and brain damage on the amount eaten and how hard a rat will work for food. These effects were quite well summarized by a simple, static feedback model whose central assumption is that animals regulate 1 eating rate. Chapter 6 did not deal directly with interactions among multiple motivational systems, nor did it deal with the regulation of activities like running or lever pressing that do not fit into the traditional motivational trinity — hunger, thirst, and sex. It turns out that both these omissions — motivational interactions and “weakly” motivated activities — can be handled in the same way; that is, by an extension of the regulatory approach. The last chapter argued that animals seek to regulate their rate of eating. This is hardly controversial: Food is essential to life and failure to regulate food intake, at least over the long term, does not favor a large posterity. But experimental results suggest that animals do more than this: They regulate even over periods of time too short to pose metabolic dangers, perhaps because of the need to anticipate the possibility of deprivation in the future. Moreover, eating is not the only thing regulated in this way: Essentially everything the animal does tends to occur at a certain rate, and animals will take action to restore this equilibrium rate if it is perturbed in some way. Before I can explain how this works, I need first to explain some basic economic concepts, such as utility (value) and marginal utility, and the idea of a constraint. I begin by describing a typical experimental situation used to study multiple activities, then describe a fundamental constraint to which all behavior is subject: limitation of time. I go on to recount the pioneering work of David Premack, who first experimented with “weak” reinforcers and proposed a general definition of reinforcement based on this work. Optimality (economic) models of operant behavior derive, in large part, from his proposal. The remainder of the chapter describes the optimality approach and shows how a wide range of experimental results can be accounted for by two principles: that the marginal utility of any activity decreases with the amount available (the principle of diminishing marginal utility), and that animals act to maximize utility, subject to constraints of time and reinforcement schedule.